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ABSTRACT 



A study examined how research concerning an educational 
innovation such as service learning can be enhanced by the involvement of 
practitioners. The context was the Service Learning Impact Study, a 
multifaceted 3 -year study of the impact of the Helper Model of service 
learning on a target population of 1,400 middle school students for whom 
service learning was required. The study examined the impact of participating 
in the Helper Model during early adolescence, identified how impact is 
related to program characteristics and to the types of service students 
perform, and developed a preliminary framework and exploratory hypotheses 
concerning the relationship between reflection and the service learning 
experience. Data were collected from the following: service learning 
participants and 400 comparison students, teachers, program leaders, and 
school administrators, using surveys, interviews, and observations. Work with 
the Impact Study indicated that empirical studies of such educational 
programs must find ways to work with practitioners because without the 
involvement of teachers and students, important knowledge that can inform the 
research and practice may be overlooked, missed, or misinterpreted, and data 
may be inaccurate . Three key ways were found to work with the practitioners : 
establishing a teacher working group, maintaining relationships with schools, 
and collaborating through data collection. Students were involved through 
approaching them as the experts, establishing a student advisory group, and 
serving as a service site. (YLB) 
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Lessons Learned from A Study of Service Learning 1 



Deborah Hecht & Dana Fusco 
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The purpose of this paper is to examine how research concerning an educational 
innovation such as service learning can be enhanced by the involvement of 
practitioners. This paper is in many ways a “work in progress.” As we work with 
practitioners we continue to be convinced of the importance of their input. We also are 
continually finding new ways to work together to improve our study and make meaning 
of the findings. In this paper we discuss how our research has been improved, 
enriched and informed by practice. 
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Service learning has received a great deal of attention in recent years. Advocates for 
alternative approaches to juvenile justice and youth program specialists, as well as the 
President and the United States Congress, have endorsed service as a way to 
reconnect youth to the community, motivate youngsters to learn, provide needed 
service in urban areas and help students become active learners (e.g., ASLER, 1993; 
Hedin, 1987; ConradS Hedin, 1991; Perrone, 1993, etc ). Evidence suggests that 
students who engage in some form of service activity gain knowledge about the activity 
and themselves, experience an increased sense of competence, hold more positive 
attitudes about the community, and have a greater sense of responsibility (e g., 

Hamilton & Fenzel, 1987; Hamilton & Zeldin, 1987). Despite interest in and adoption 
of youth service programs, there is limited empirical evidence to support the benefits for 
students of participating in service learning. While policy makers and researchers 
generally agree there is a need for more rigorous empirical studies to document how 
the service experience impacts upon students, practitioners often report that they are 
already convinced that service learning has a positive impact. They often rely upon a 
single case study or first hand accounts of positive student change. Much of this 
anecdotal evidence comes from teacher descriptions of students and programs. While 
these data provide a program-based view of what occurs from the perspective of those 
actively involved in the program and present very persuasive argument for service 
learning, they are usually somewhat limited in scope, presenting data on a very small 
number of students. 



1 Funding for this study is provided by The William T. Grant Foundation. 
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In contrast, many empirical studies collect information from a large number of students 
across varied programs. Typically they have relied upon survey data collected from or 
about students using a pre-post design. Often “exemplary” programs are sampled 
since such programs are believed to maximize the change which can be observed 
among students (e.g., Newmann & Rutter, 1983; Conrad & Hedin, 1991). Program 
characteristics such as the length of time at a site, type of sites, and basic school 
characteristics are usually collected and reported as background data. There have 
been some attempts to include this information within the analyses but these have been 
limited by the tremendous variability across programs and the availability of 
sophisticated statistical techniques that can handle such data. Overall, the results of 
the empirical studies have not been conclusive leaving researchers to seek new ways 
to collect and evaluate data. 

In this paper we propose that empirical studies can be improved by involving 
practitioners in the process. Research on innovations in education, and the move 
toward action research provided us with some guidance. They suggest that real 
change occurs only when those involved in implementing the change are also involved 
in the planning. Empirical studies of educational programs such as service learning 
may also find that including practitioners in meaningful ways will improve the quality of 
the data. As a basis for this exploration we examine our own study of service learning 
during the middle school years. Although this is not an action research project in which 
teachers are the researchers, we have worked to incorporate the practitioner’s 
perspective within a more traditional methodology. We believe that for a study such as 
this to be meaningful it is important to involve teachers and students. This involvement 
requires not only their participation in the required data collection tasks, but also their 
involvement in the process. They need to participate in the development of 
assessment tools, defining a methodology and interpreting the results. This paper 
discusses how we collaborated with service learning teachers and service students. 

SERVICE LEARNING IMPACT STUDY 

The context in which we consider these issues is the Service Learning Impact Study , a 
multifaceted three-year study of the impact of the Helper Model of service learning. 

Pilot work for this study began in 1993. The Helper model requires a structured service 
involvement lasting a minimum of 10 weeks, weekly reflection, and a specific person in 
charge (called a Program Leader) who is actively involved with the students and 
remains involved throughout the students’ service experience. This research is 
exploring whether and how participation by middle school students in a Helper Program 
contributes to the development of academic or school-related skills, enhances 
psychosocial abilities, and leads to the acquisition of skills for the school to work 
transition. The specific goals of the study are: 1 ) to examine the impact of participating 
in the Helper Model of service learning during early adolescence 2) to identify how 
impact is related to program characteristics and to the types of service students 
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perform, and 3) to develop a preliminary framework and exploratory hypotheses 
concerning the relationship between reflection and the service learning experience. 

The sample includes approximately 1400 students from six middle schools with 
Service Learning Programs based on the Helper Model. At each school service 
learning is a requirement for all students within a given grade. The two urban, two 
urban fringe and two suburban schools include an ethnically mixed student population. 
Each program meets the essential features of the Helper Model, including weekly 
reflection and a minimum 10 week service experience. Comparison data is being 
collected from over 400 students not involved in service learning. 

Data are collected from service learning and comparison students, teachers, program 
leaders and school administrators, using surveys, interviews and observations. 

Student surveys that include forced choice and open-ended questions are collected 
before and after service learning. Using a combination of qualitative and quantitative 
methods, the impact of service learning on students and differences in impact 
according to program characteristics are being investigated by comparing student 
responses over time, contrasting service learning students with comparison students, 
and examining differences across sites and programs. 

Throughout this study we have looked for ways to collaborate with teachers and 
students. The findings that we report are based upon our notes, observations and 
experiences. The evidence, while mostly anecdotal, represents the start of a 
framework for thinking about how practice can inform research to produce more 
meaningful results. 

RESULTS 

Our work with the Service Learning Impact Study has convinced us that empirical 
studies of educational programs such as service learning must find ways to work with 
practitioners. Without the involvement of teachers and students, important knowledge 
which can inform the research and practice is likely to be overlooked, missed or 
misinterpreted. In part this is because service learning is not a specific program or 
curriculum that is uniformly implemented within a school. Rather, it is a general 
approach to learning which involves students in real-world activities within their 
community and therefore is unique to a given setting. Furthermore, it is continuously 
changing. Staff changes, schedule changes, the addition or elimination of sites, funding 
shifts, etc. can have a tremendous impact or very little impact on a service learning 
program. Although research which examines any school-based program must remain 
alert to such changes, studies of service learning are almost guaranteed to encounter 
these difficulties. It can be speculated that exemplary service learning programs, the 
focus of many studies, have staff who are especially adept at assessing which features 
work best for their school and then adjusting the program as necessary. The 
involvement of practitioners in a study is one way to remain aware of such program 
changes. Our experience indicates that changes which the research team considered 
very important (e g., elimination of sites, changes in who conducted reflection sessions, 
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more frequent meetings with site representatives) were often viewed by the schools as 
“part of running the program and hardly worth mentioning.” 

Another reason it is critical to involve practitioners concerns the accuracy of the data. 
Studies of service learning often rely upon self-report data from the school to document 
program characteristics. The researcher may interview program leaders or ask for a 
survey to be completed by someone knowledgeable at the school. However, research 
indicates that even when these reports are carefully completed, they may not clearly 
describe the program characteristics. If the person answering the survey is not actively 
involved with the program, the information may be based more on the program’s 
mission statement than on practice. For example, to ask “what types of reflection 
activities are typical,” is rarely adequate. Although a “reflection” period may be 
scheduled and a list of activities made available to the researcher, more details are 
needed to fully characterize the sessions. Reflection may not be linked to the service 
experience, the reflection leader may have different goals for the session than the 
program leader, the students may not understand the sessions' goals, etc. We found 
that unless data were collected from multiple sources concerning reflection and site 
activities, including observations by the project staff, it was difficult to assess the 
characteristics. 

In summary, we believe that unless a study of service learning finds ways of 
communicating with practitioners and finds ways of involving them in the process, the 
findings of these studies will be limited and possibly misleading. This belief was further 
reinforced by experiences with one of our study schools. The one school which was 
included without adequate time to establish a strong relationship eventually dropped 
out of the study. 

In this section we highlight key ways we found to work with the practitioners. From the 
start we were committed to including program leaders and school representatives in the 
process. Recently we began to include students more fully and are developing new 
ways to work with them. 

Involvement of teachers and program leaders 

From the beginning we believed that it was essential that the researchers establish 
close relationships with service learning staff at each of the study schools. We meet 
with them several times during the development of the study’s design. Since we would 
be requesting a great deal of information from each school we felt that it was important 
the school staff come to see us as partners, rather than an outside entity only there to 
collect data from them. 

Establishing a teacher working group - The Research Planning & Liaison Group 

One way we established a strong, collaborative working relationship with the program 
leaders was through what we have called the Research Planning and Liaison Group. 
RPLG . This group is made up of teachers and representatives from each service 
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learning program which is participating in the study. The group meets with project staff 
two to four times per year and has been involved in the planning, implementation and 
development of the project. Members critique all materials and have direct input into 
how work continues and we rely upon these meetings to guide our work. Although we 
found that logistically, getting six teachers together was very difficult, it was not 
impossible. We also found that providing food and reimbursement for transportation 
was a valuable incentive, while providing a stipend was not. 

The group met for the first time when we were piloting survey forms and the research 
methodology was still being designed. The agenda from this first meeting was to get 
participants to think about and suggest ways that the research might be useful to them. 
Three key questions were presented to the group for discussion. Participants were 
asked to “think back to when you were beginning a service learning program. What 
types of information from the research would you have found useful? How can the 
research results be used by you to help evaluation and improve your program? What 
would you like to learn from participation in this research project?” Clearly the 
message we hoped to give at this meeting was that the research team valued the 
opinions of each program representative and that we wanted the research to be useful 
and meaningful to them. 

This meeting, like all others, was also a time to deal with logistics. At almost every 
RPLG meeting we spend some time discussing the “How & When”— When is the best 
time to do student interviews? To collect surveys? What should the instruction forms 
look like? By discussing these issues as a group, participants were able to hear how 
other schools handled our requests and collectively we often developed new plans or 
procedures. This was especially helpful during the initial stages when we were still 
working to establish trusting relationships with the schools. 

Over time these meetings become more focused. We would review new survey forms, 
asking for teacher input before they were administered. We would also provide 
feedback from our work to date. The group gradually become a network of service 
learning professionals who were able to share experiences beyond that of the study. 
Although the specific individuals in the group have changed somewhat due to staff 
changes, a retirement, the addition of new schools to the study, the group has 
remained committed to seeing that the research is accurate and relevant. 

Despite the differences across schools, the participants have been able to examine 
differences and similarities across programs and often borrowed ideas from one 
another. For example, among the highlights of one meeting was an informal discussion 
between representatives of two schools, with markedly different enrollments (one 
primarily Dominican, inner-city; the other white, working class, and less urban). They 
found the differences between the school intriguing and began planning an exchange 
in which students from the two schools would jointly execute a full day’s service project. 

At other meetings, representatives have shared reflection activities, hints for locating 
sites, and dealing with public relations questions. We are now at the point where our 
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meetings explore specific topics - the last reflection. We spent time discussing what 
happens during reflection and their thoughts about how we should study the topic. 

In summary, the Research Planning and Liaison Group meetings have proven very 
useful for informing the research about practice as well as for informing practice about 
the research. However, this group alone was not enough. 

Maintaining relationships with schools 

Establish strong personal relationships with service learning staff : We found the 
relationships with the schools were most productive when they were personal. We were 
once told the best indicator of a strong relationship between a service site and program 
leader was when the program leader knew the site representative’s home phone 
number (Halsted, personal combination). We found a similar relationship for the 
research. Based upon our experiences, the key person who needs to be involved in the 
study is the program leader, not the principal or other administrator. The Program 
Leader is the person who oversees and actually manages the day to day operations of 
the service learning program. This might be a teacher, a guidance counselor or “a 
service coordinator.” If the person who carried the official title of coordinator left most 
of the daily program management to another teacher, that teacher became the contact 
person. 

Frequent school visits: We found it was important for the researchers to visit the 
schools. Phone contact, as used during pilot work with some schools, was much less 
effective in keeping schools engaged. This was reflected in less complete data. 
Especially at schools where more than one person was involved in the service 
program, being a visible presence at the school was especially helpful. (The schools in 
our study vary, from all teachers being involved in service learning - taking students to 
sites and leading reflection session, to only one teacher.) 

Collaborating through Data Collection 

Project staff must be present : We found it was very important that data collection be 
supervised by project staff. Throughout the study we tried several procedures for 
collecting surveys at the schools. These included training teachers in administration, 
providing structured instructions to be read, and modeling administration procedures. 
Even when the teacher was familiar with research methods and was very willing to 
help, there were serious problems. Both the quantity and quality of surveys increased 
dramatically when they were administered by project staff. Additionally, by being 
present during the administration we were able to listen to questions which students 
asked, helping us to better understand student responses and to revise the survey 
when necessary. Furthermore, our experiences indicated it was not only having the 
surveys administered by the project staff, but the relationship with the classroom 
teacher that lead to greatest success. 
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Meeting the scheduling needs of the school: We found that it was very important to 
consider the scheduling needs of the school in preparing for data collection. For 
example, in New York City students take a large number of standardized tests during 
the spring semester. By working with each school individually to schedule data 
collection we were able to avoid overwhelming either students or teachers with testing. 
We also found our beliefs about howto “best schedule the administration” were often 
wrong. Since the survey is long, we tried to administer it over two days. However, the 
teachers and students quickly told us this was disruptive. (Incidentally, this was the 
last time we made a decision such as administering data collection over two days 
instead of one without consulting the teachers and students.) 

Attending to the survey language: By establishing a relationship with each school we 
were able to assure the language in the survey was meaningful to students and teacher 
at that school. We found that the study schools differed greatly in how they defined 
service learning, reflection, sites, etc. Although all are “Helper Programs” they applied 
the model in somewhat different ways. For example, at one school the reflection 
session is called advisory, at another school it is called seminar and at still another it is 
called reflection. When we tried to use a generic term for reflection students were often 
confused. Had we not established communication with the schools, we would have 
been unaware of this confusion. (We found students would answer the surveys, even if 
they did not understand the question.) Thus, it was important for the language of the 
survey to match that of the program. Furthermore, we found it was problematic if the 
pre-surveys asked about service learning when the students were not yet familiar with 
program. Our relationships with the schools allowed us to be aware of what students 
knew about service learning before the class began. 

Involvement of students 

Although students are typically the focus of studies of service learning, they are rarely 
included in the planning and development of the study. We interviewed students 
throughout the development of this project. By interviewing even a small number of 
students at each school we demonstrated our willingness to “listen” to what students 
had to say. Students reviewed surveys, provided input into interpretation of the results, 
and offered suggestions for the research design. We found that when the surveys were 
administered to students who did not feel that they had an investment in the study there 
were a large number of incomplete forms and patterned responses. We are continuing 
to seek ways of meaningfully collaborating with students, some of which are describe 
below. 

Students are approached as the “experts” 

Whether administering surveys or conducting interviews we always explained we were 
there because the students were the experts and we needed their help to understand. 
Students were encouraged to write comments on the survey or ask questions of the 
research staff who were present during administration (allowing them to feel a part of 
the study not just the subjects). Many students commented about the wording of 
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questions ("what does outgoing mean?") or the format of the survey (“it’s already asked 
that.") Others explained their responses (“this happened to me”) or suggested 
additional questions to ask (“how do you think your director felt about you and your 
work?”). Student comments have been used to revise the survey and interview 
questions. 

Establishing the Student Advisory Group 

Another way that students are participating in this study is through the establishment of 
a Student Research Advisory Group . This was recommended by members of the 
Research Planning and Liaison Group and is now under development. The Student 
Research Advisory Group will consist of two to three students from each school. The 
group will convene at least once a year to discuss the project and to help interpret 
results. They will also develop an independent project, documenting the impact of 
service learning through student eyes. Our first meeting is scheduled to be held in May 
1997. 

Serving as a Service Site 

Another way we have involved students and been informed by students has been as a 
service site. One of the schools in our study includes “office experiences” as part of 
their service program. Every week three students work in our office for two hours. 

They assist with a variety of tasks, including data entry, reviews of surveys, phone 
calls, etc. We found this was an excellent way for us to understand more fully what is 
required as a site. It has also provided us with an opportunity to better understand how 
the students think about the study, the questionnaires and our findings. The students 
have also been very helpful by informing us about what are “really” happening at their 
school. We discovered they knew which students had switched sites, who was not 
happy in service learning, and who was skipping their service sites. 

EDUCATIONAL IMPORTANCE 

We believe the importance of this work is in demonstrating that empirical studies of 
programs such as service learning require a different approach than typical of many 
other subject areas. Our experience suggests without open communication among 
teachers, students and researchers the study will be less successful and the findings 
less meaningful. As more studies of service learning are proposed, we believe it is 
critical that researchers carefully consider not only their design and instrumentation, 
but also how they can establish these needed relationships. 
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